Nonlinear dimensionality reduction of high-dimensional data is challenging as the low-dimensional embedding will necessarily contain distortions, and it can be hard to determine which distortions are the most important to avoid. When annotation of data into known relevant classes is available, it can be used to guide the embedding to avoid distortions that worsen class separation. The supervised mapping method introduced in the present paper, called ClassNeRV, proposes an original stress function that takes class annotation into account and evaluates embedding quality both in terms of false neighbors and missed neighbors. ClassNeRV shares the theoretical framework of a family of methods descended from Stochastic Neighbor Embedding (SNE). Our approach has a key advantage over previous ones: in the literature supervised methods often emphasize class separation at the price of distorting the data neighbors' structure; conversely, unsupervised methods provide better preservation of structure at the price of often mixing classes. Experiments show that ClassNeRV can preserve both neighbor structure and class separation, outperforming nine state of the art alternatives.

Weaknesses: I have the following critical concerns about this paper: 1. The used datasets are too simple. More complex datasets such as SculptFaces in NeRV paper are required to evaluate the performance of the proposed algorithm. As detailed above, ClassNeRV could be seen as a variation of NeRV through penalizing within-class missed neighbors and between-class false neighbors with class information. Therefore, in my opinion, it is not a significant contribution. According to Sec 3.2, they derived the ClassNeRV Stress Function from NeRV Stress Function by splitting Eq. 2 into within-class and between-class relations.

Three referees indicate accept, one indicates that the paper is marginally below threshold. I agree with reviewers 1, 2 and 4 that the presented approach is insightful and useful to NeurIPS applications, and support an accept after reading the rebuttal. However, when revising the paper, please take into account reviewers' concerns about improving quantitative comparisons with other similar methods as well as providing further discussion. Please consider adding experimental support with more complex data to the the main paper or Supplementary Materials.

Nonlinear dimensionality reduction of high-dimensional data is challenging as the low-dimensional embedding will necessarily contain distortions, and it can be hard to determine which distortions are the most important to avoid. When annotation of data into known relevant classes is available, it can be used to guide the embedding to avoid distortions that worsen class separation. The supervised mapping method introduced in the present paper, called ClassNeRV, proposes an original stress function that takes class annotation into account and evaluates embedding quality both in terms of false neighbors and missed neighbors. ClassNeRV shares the theoretical framework of a family of methods descended from Stochastic Neighbor Embedding (SNE). Our approach has a key advantage over previous ones: in the literature supervised methods often emphasize class separation at the price of distorting the data neighbors' structure; conversely, unsupervised methods provide better preservation of structure at the price of often mixing classes.